{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The XOR dataset\n",
    "=========\n",
    "\n",
    "The XOR dataset is a non-linear dataset that is based on the [Exclusive OR](https://en.wikipedia.org/wiki/Exclusive_or) logical operation. Given two inputs and one output, the output is equal to 1 (true) only when the two inputs have the same value (both true or both false):\n",
    "\n",
    "- 1 XOR 1 = 0\n",
    "- 1 XOR 0 = 1\n",
    "- 0 XOR 1 = 1\n",
    "- 0 XOR 0 = 0\n",
    "\n",
    "The **XOR affair** was one of the reason that lead to dismiss research on neural networks in the Seventies. Altough there waere multiple factors that influenced this phenomenon, one of the main reason was the publication of the book called [Perceptron](https://en.wikipedia.org/wiki/Perceptrons_(book) by Minsky and Papert in the 1969. In the book was proved that a simple neural network (like the Perceptron invented by [Rosenblat](https://en.wikipedia.org/wiki/Frank_Rosenblatt)) was not able to solve non-linear problems like the XOR classification task. Only in the Eighties it was discovered that using a multi-layer version of the Perceptron it was possible to solve the XOR problem.\n",
    "\n",
    "Here I will show you how to build the XOR dataset from scratch using Tensorflow and how to store it in a TFRecord file that can be used to train and test our models. The idea is to generate random points in a **Cartesian plane**. We need two input values in order to apply the XOR operator. We can use the X and Y coordinate of the plane to represent those inputs. Each point has an associated label, in our case we represent true with a positive number, and false with a negative number. In this way we have an input tuple of two values and a single output representing the target."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "import tensorflow as tf\n",
    "\n",
    "interval = 10 #the interval used to generate the points\n",
    "dataset_size = 2500 #elements in each quadrant (total = dataset_size * 4)\n",
    "\n",
    "#those are the portion of the Cartesian plane having equal-sign coordinates\n",
    "input_up_right = tf.random_uniform([dataset_size, 2], minval=0, maxval=interval, dtype=tf.float32)\n",
    "target_up_right = tf.zeros([dataset_size, 1], dtype=tf.float32)\n",
    "input_down_left = tf.random_uniform([dataset_size, 2], minval=-interval, maxval=0, dtype=tf.float32)\n",
    "target_down_left = tf.zeros([dataset_size, 1], dtype=tf.float32)\n",
    "\n",
    "#the up-left quadrant has positive Y and negative X\n",
    "input_up_left_x = tf.random_uniform([dataset_size, 1], minval=-interval, maxval=0, dtype=tf.float32)\n",
    "input_up_left_y = tf.random_uniform([dataset_size, 1], minval=0, maxval=interval, dtype=tf.float32)\n",
    "input_up_left = tf.concat([input_up_left_x, input_up_left_y], axis=1)\n",
    "target_up_left = tf.ones([dataset_size, 1], dtype=tf.float32)\n",
    "\n",
    "#the down_right quadrant has positive X and negative Y\n",
    "input_down_right_x = tf.random_uniform([dataset_size, 1], minval=0, maxval=interval, dtype=tf.float32)\n",
    "input_down_right_y = tf.random_uniform([dataset_size, 1], minval=-interval, maxval=0, dtype=tf.float32)\n",
    "input_down_right = tf.concat([input_down_right_x, input_down_right_y], axis=1)\n",
    "target_down_right = tf.ones([dataset_size, 1], dtype=tf.float32)\n",
    "\n",
    "#now we can concatenate (vertically) the four quadrants into a dataset\n",
    "xor_features = tf.concat([input_up_right, input_down_left, input_up_left ,input_down_right], axis=0)\n",
    "xor_labels = tf.concat([target_up_right, target_down_left, target_up_left, target_down_right], axis=0)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To verify that everything is correct, we can use Matplotlib to plot the dataset as a scatterplot."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXwAAAEACAYAAACwB81wAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJztfWuMZkl53lPT3bsjcAshxyzmsovHnSwz8ymb8GMZQDOM\nRIDFUmQSW5ET+ZZYTiSEIsX+wSL/GG3kH05kGZZElmbGCDmSEUp+OLuOANMt7zDjTLOLtKyhZ5YB\nFO+u+RYvieTo+xwZTA+VH6fPdHVN1Tl1eavqrfPVI7X6dr5Tt7ee91JVbwkpJRoaGhoapo9jpSvQ\n0NDQ0JAHjfAbGhoaVgSN8BsaGhpWBI3wGxoaGlYEjfAbGhoaVgSN8BsaGhpWBCSEL4T4pBDiFSHE\nV5W/vVYI8QUhxC0hxB8LIV5DUVZDQ0NDQxioLPxPAXi/9rdHAexIKR8E8CcAPkpUVkNDQ0NDAATV\nwSshxAMA/khK+fcPfv86gHdLKV8RQrwewBUp5VtJCmtoaGho8EbKGP7rpJSvAICU8i8BvC5hWQ0N\nDQ0NI8i5aNtyODQ0NDQUxHrCd78ihLhPCel81/SQEKIpgoaGhoYASCmFz/OUFr44+OrxJIBfPvj5\nlwA8YfuglLLur8UC8vr17nvhci9cuFC+P2r9un4dcn0dEoDc2IDc3SXpz8VC4qGHJDY2uu+LRdp2\n5C7P9asG2bx+XWJ9XQLo+m93t3ydbF8hoNqW+WkA1wH8PSHES0KIfwngtwC8VwhxC8B7Dn6vA8sl\nsLvbfXd59uxZ4Ny57rvLZyiQq1yfvuCEkHrPZsDp08DGBnDqVPczATY3gWvXgKtXu++bmySvJSmv\n1uFNhUQiwAYkhC+l/BdSyjdIKe+VUt4vpfyUlPKvpJT/SEr5oJTyfVLK/0tRVnL4EuneHnDjBrC/\nD9y82f2cok76rMxVbgqlkpplQuudkJk3N4EzZ9KTvU95pWyVUKhik0qEcivn7GDglkhvLBZSXr/e\nfafG9etSrq9LCUi5sSHl7u54XR56qHv2oYfo69S/f3396Pst5T711FN0Zfv2hQts7fF9x9D4D9Xb\nU3b0/kwpei6gLD/F8A4hRjZVsZnNuq8YEQopv+S4m3DAnX586/sB6i9vwqcgDJf3+xD4YtHNltwK\nKKZcFwlOocxiWcZl/G31jpSd1KKXu/zUtkosVBFVxWZtLVyEQoi79LjbsBqEn8MsSUngIXWhnpU+\nEkzdF7HtcR1/U70jZSe3RZyj/FKiPka8uojO54di01v4viIUStylx92G1SD8VGYJR5+th2lWxtRX\nN5d2dujq6oJYzyR0/CNlp7RFXLp8KrgQr4lkVbEJEaFQsefa76tB+FKmszq5+Ww2xNZ3sehMJKD7\nms34t1lFrMKIkJ3FoiOK7e0yXcbJ+QyFi8WcyrENFXuO/T59wk9lhafy2TjXd3vb/A7Ono4vXOIG\nnm2tzTbgiDEy74dlPqcnWZvY14hpE37KmcYhTu5DPBT1Nb1jSmw21pbAtnKN59YGm8WcWgS5hGco\n7KppE74+0y5doid9SnPCZ3ExRMIp6qu/wxY4rdHiH+v/QObmQhhDqHXIpFyNPRlUSm3ahK/OtOPH\nu1UXrjNOSndmcA1o5pjBep37rREhklmadVziBiYPx6HOpQljCLU7aTUoVBtcRZ5KqU2L8E29t1h0\nlv3amltvcSCdMWZwJaZcM1itc6hkcmGdsf5X/8+lzpGYQsiJs0K1wTeCS6HU6iV8nZiHes+1t2qa\nwEMSnnMG28bBVzI5eS2uCOhnnyaUctK4dO/UEXJAP1ap1Uv4OjGP9Z5Lb03B1JEy3wy2KcgQySzt\ntYSwq2c/h1h0JZy0hjwooWjrJXzTQmGKXSi1wGRpU2z+HiJCagVZymuJYVcPpvRpwlRsj4Zh5Fa0\n9RK+iZgpeq9GU8dEWBQm4tg7YhSkr0WdUhlnYlefJtRsezSMY0z8U4Xz6iX8Gok5Fba3716Udglx\njUmUa1w9NHxTYlvpUH0ysKtPE5qIx4Hbkk8PVzsqRTivXsJv6LBYSHnqlLxz9vvee7utkUMk5ipR\nqYiQY7yisStLhJI25/0XiY57OCGE8HNeYt4whr094BvfOPx9fx946aXhWxlcL0JJdbNDySuCbLdg\n5L5txIJ2m9QhYi5byXHXTyjGxJ/dDVq+GoL6C83CP8R8LuWJE4cW/lh2p8WiCwHpuWJjTKnQz+W2\nqAuZfa5dlLJ6XMMbQ4ixdLmvgYyJ/9D/Y8YSLaTjAW6zRmWIEyekfPJJ971+s1m3iydmgZez32xC\ngVCSTxelzMdX0zD1iCVtqj0cXKf8WDTWVO9G+K7gOGt8GcL2fCjTcIzFD0FlkK2tzjvy/bzn7Pfp\nolVaMnFFyaUVqilPqTR80kSb6t0I3xUcZo0uOb4MYXs+lGm4+80mzOcd2fvmVdJn0XzunEPHd4hC\nCG6IVCiHiZvFmxIUU57aTnQZy6F6N8J3RUlyU+Pusadabc/HME1Nu1uuXz/cwrq+HubNrK93SsNx\nFqfuIhdSoQpvlHBySykZiinv6+G5rvWMxf9t9a6X8HNJgVpOyYXGnqRq9Ms5YT7vMqcC3XfXsI4e\nDirt7SnI5XyWcHJLR1Jjp7yr0kjhCZjqXS/h55CC0tIm5dFZ1luXueqSQ6nmNt9it37s7h69HZtB\nKCuX81nCyeUQSY3FkFPdi36udtZL+DluPLh4sby0qbNM3VmTq9yUCqaEQqViLWahrFzVyd1sHws5\n1G4oETIyLQnlUKb1En7K3lHDKEJIb/efGiXIJYfJMVRGylnIjKxzo7aFV9eYdcjNoKWceNtFcZTb\nSE3jXC/h670ztk3BR8L1MIrvAl8uxM7csT4bUqoUrGErY2wW+qxu1cRsnghpXugmJc6IuRm0VMgo\nRXhMbd9sZt7jUS/h21oamjfG9L6NDSnvuYfn7Ig1TVw+b1OqMdcYjpUh5bjlbyqbi+lGDBuph4r1\n1tZRG2Znp5xO9FFYMbZJD5tVXXLzXaorsdfXzZf8TYPwhwgiVIWrplDIIZ3UiDVNTBk2h6AyTN8v\nqcyioVlo2lbJyXQjxBCphzRPd1xPnDBbgVR1H0v/6xOGodh2OuRQcozw+Xpwavt6C19v6zQIvyfn\nocyQKa7cK4kY02Sx6KTBNf+OlHf3h62/qWCbhaZtldxMNyK4ODo+zVM/s7Ul5RNPhIm4K5kP2Uql\nLoPxIXeT05jLGwp1UNX2mdpaP+G7SFeICs9FGDFSNNSuoffqvt/OjltZan/M53eXnWNG+JA7V9PN\nEb0dM7SUESLW+h3sIUpjiIhUJwzo2mALSbnsvjHl+ksNvZ2UUUwXpLI36yf81FfflT4imeK9ocps\nTMFQtMXVfJwYuevQI2ipIoq+3eaay0VdKwi9O0ftg5w7kqW8u52XLuV1+FPZm/UTfs2ueyplleqm\nqtgyx+CqNHKSe04/XkGK64IpmuE63YairK4oGVU1ObRqOCzHkl4KMa+f8KWM75lCk9ooVTlnJSUo\nyqQO1lJtGy2w04dyCKmb4TrdTM/5DEtpW06v/xS2tE6D8GNQcFLfKV89rp97VlKCQvFSzHCqMfXZ\n4J3AYKAaQi77D0KGxdYH1F3u8r6xRfTt7e7LRRGW2grbCJ+L7xyb42UqB4woWI5qTF0UkAOL+ZBB\nCowt/uYC9bBQpxx2iSTalo9cN72Vti8b4XPxnWMWUidwwIgU1GM6pIBGWMyHDFIgZPE3lf1ANSzU\nNppvCmPTOUF1V9LQofzYuseOTSN8Kfn4ziH14OKvc0OukNYIi/mQQQr4ikdq+4FiWKhj+7HvC7Hw\nQ4/PxI5NI3xKmEYzdbil9MoWd+QId6ksppXHxcJ3FY9a7AdqfU6x/LSz47Z1NLQs1813Q+LeCJ8a\n+uSfzToTj3Km66MaKkFTiv2b4GMSufTF2DOW8nzIIAV8xKPZD3wxNjYu4t4IPyW2tw9NO8DtROsY\nqHzu2mP/LgQdk0Yx5JlazOMR5IqGpUJKO6a0jTQ0Ni7iF0L4x9BQDnt7wI0bwP4+cPNm93Ou9yyX\nwO5u970klkvg7Fng3Lnuu60+sxlw+jSwsQGcOtX9bIJLX7g841oec2xudlXf2ys/1L5wFY2S746Z\nRkNjk0z8fDUE9RdqsfD7kE5/NpzCLKDyuX3fw8kjiN1WYQqJuW6/dEn+wtA89rFMXR0ejtHAlE5W\nzsPkoZ/vw4e2LcBoIZ3ESEEAVO8ce486qzmFK1JsdXDpU6ZkPgZfkhkb6qH3lVQEi0XaRGsUtlbs\nNIoZGymnRvhczY4aoUsOp4u7+5kdsgrKSXElgj4NQrdm2oba9r6cTqDNSUudaI1iNw/FFlDbPpCx\nsZ4O4VNIW1MYhzClUOZg4VL5xAUVVwox699pytAR0uShoba9L5cuNYlASMbvUoiZRmNR4rGxng7h\nUxx64hKjpkYIw/SSVWoDuVqPGHPV9k7XUJbL3x2r3v+NWszUd9ouI6PW1balkRS7kHXYrkTgIK6p\nEZMId7FgSvgAXgDwZwC+AuAZw//NrYyx3Kbq6scwjO81iC51CbmzzdVcpTCdh2L8Hv1oezyFmOW+\njEyF2uVj1idlmabzjY8/LuWxY/F9y9nRD6W5/nNcCf9/AXjtwP/trYrxlbjEqCkRm5SNMieNr+IZ\nChabzEsK09lWpmc/jsW5QyasjYT0d5ouI/OBK+HpXb69nc9mUkVArcfx4/FR3RxeSgxCaO5QHnkS\n/p8D+NGB//v1js9+tNIxampQrBKVyjPkU/fUGTI9+9FmhfZxdp8uddFlscM0tAZgg97lOztlbCa9\nHpcvh5ed4qyklOW9hhos/GcBfBnArxr+79fKKcblfWCziHNKYIxp63rbBqU3YguCerCqzQp1rd58\nLuXFi+GXjbvCZQ1g6HO6UitxDQPV0HM+HE9RjxDCF7Ij3WQQQvy4lPI7QogfA7AN4MNSyj9V/i8v\nXLhw5/nz58/j/Pnzd79od7c7Fre/3x0/u3oVOHMmad2TYbnsjtfNZt1xu9h3nT3bnRY9fRq4di3+\nnS5lPv00IATw8MNpylsuD9ukv9+1/yj7WYGvKL78MvCTPwl873vAvfcCJ04A3/pWd4KSerj0uj3w\nAPDii25l2bo8UTd61yPkPe98J/D1rwNvfStw/Xp8/UvS0JUrV3DlypU7vz/22GOQUgqvl/hqiJgv\nABcA/Jr2N3eVNoW4PLWJkHuBOrT+VF6Ia/kJTTFfUbx48ail+YlPpLOcU6wBmNbauS6E6qD2UjjR\nELiFdAC8CsCPHPz8agD/E8D7tGfcW5jKx8wpwSlu5QpZmQptc0z8noJ8XctPrAh9RHE+7xYgge57\n6kuzKaeJLbbve7VhLQqih63Oi8XhOUGqa6tDwZHwfwLAc+i2ZH4NwKOGZ9L1iAtyB+WoTYSe8H32\nz8W0OaT+1Beau5TPyRSTHTlcvuxO9i4kmYNI9W703b3DJebtA1ud1b/PZlKePFl2BxA7wneqgAvh\np5Rsyh0hJXYQhdQ/9vYF3/qnUHKui78V7tRy3cmTi0hNC9Y1XMJC7cSqf+/PCFDvAPLBNAk/tWRT\nkFFJM8YUtHUxDYfanKI9lZJvj36XTepwjJRHr1K0XaNYmkg5XsKiEnwKJ1b9+4kTeQnfpLzqJfwh\nVZxDsmPJKFcdxyxun43XQ22e6knlQOSOwavl3XOPlLdu3f0Ms4jVIHLoep3gYw+O2eqsTjXXSGps\ngMKmvOolfDUwpid/rkGyU9fR1VwZIupe6ig8gBWDvsvm8mWa99qIQB1GoNtPTxFZGyqzdpQ4OObS\n//0SW0ys3zat6yV8VbrX1u4eoZzhgNAZkbKOvtf72XzR/ry6qY9ztqcypLDwh3T4YtGRfD8lbGEd\nyjLVZ2pUCCbRLy3Ci0WXEyg29GOb1vUSfi+Ffa+UCiOoM2JrK0/A1gU+FrdJynWTsYVqvOG7y2YM\negrgS5eODtl83okg5TJK7IUb3FGa4PW6PPQQ3eKuqW31Ev5i0fVEquttXOHqS5eAacRdzTFVYcRm\npGogge502Sz9nIeG2tINHXQqSbF9M4Twk6dWGIMQQt6pA9WZ6lAsl8Db3tadeweA9fXuPDrHFA6+\nKRX6vr3/fuCll8r1ccMdLJfAZz4DfOhD+Y7qm6ZYnzrhgQeAn/qp7l73FGkfVgn99Lx5E3jwQeDj\nH6fPQiKEgPRMrcCL8Dng5ZeBd78beOGFfLlpQjCl3EIrDJUYSpCsbjd89rPNHqBCavu1ET4VSnsa\nLijNFA1kKCluzW6wI3fSOF+EEP6xVJWpGpubndRzHOUem5sdyV+92plle3udhJbActkxR6ryKd/v\n+a7UTQPKiFvfrgce6BTNxkZnN5w+na8OnNHbU+fOdd9zTq2kMucb9Kf+QulcOhzhszeu9NaKsfJj\ntrn63uIRW1eHx2vdtqhCb1dsRs0potQCto+IotpdOrUgx2z3JfDYaw9j2zN22Cs0lbK6PZZq5kVe\nbRiSKZIjVm03ToiYx5w9jJlWPmPTCF/KdKScy5L2nY2hkknVnqHyQ5lFTSaztkZ3k7dnX+mP57zn\nNSVcuyGXN5OynBgxH9oWa6tz7LTyEdHVJvzFopuRfYILalLOZRaFEHjIhm3qlMVq+Xo4xpesb93q\nksj0R1tv3aKLOXj2lfp4jNXHDWPdkMu+SV3O0AX0oUpmqM4U08pVRFeX8PsR6K3CFKScc7bnODKY\nqj2xAeLF4mheAWamNKfTnCmRy75JXY5JzGOVzFCdcybaW13C14+1pUylPKXZHtKeMdModgZvbx89\nj87ptPMKIZd9k6McXcxjRXSI1E2prVOFrFaX8FWpmc261bVGEv4Yk0wX0yh2tWs2OyT7Eyf45DNi\ngD5qqSeUTVleiH3jQ3B9m3JOWV1E9QSyLjaN7b4C07tThaxWl/ClnJ71nRsuZO6TtXMsQDyWF3h9\nvcw1Qkyh68JS1+qNYUyM1KHPtU5gq6fpCgkXgh6zaVTxTxmyWm3Cb4iD67WHsf730Cyn8u9zbS/J\nCFMyrpBLPVJ3i88uXQ67nvT6XrpEY9Ooz6UKWTXCbwiHq2TGelIuOXpj3p/QbExJmC7RtBgLP/eu\nG5ddujkuKfGtb+jGsrEyUgQfGuE3xKHm3UE9EvnQKQnT9d2LRUeSIfHunIetbGKktrO/boJDJFav\nQ4o6pTAWGuHnwgRDBlmRcpYnUihU+6vHli5SXolc2pqWMs3FLtyRylgIIfyWPM0GWwajklmVuCE0\ny1PKbGFqUrk+gyhBNqrZLC7J2JDYxL7bBaZuKYEXX+wyj+/vd4leb9w4/F+ORHUlsLfXtdPU5uzw\n1RDUX+Bo4Q+p5FVLRGJDyS0WPiCsZ4xjMraYGbM1sSaH0+ZpUIsTpz5J5V2hhXSI4LLVoLRvXBqm\nPsoxy3zLYKKgUxGdT/yfEwEOXbtMkemDmy2SIopZL+GXJgj9GZ+NthTlp/x8Kti2N4RmqfK5m9en\nDEYKOgXR+eym5USAOkKHySQ6lMc5uE4/KWsmfPW4m37sLQYukm57hoLUU5hvnCRQ7aNQ5vLpo5gy\nSm8FsaBfxAzVRy6fZ+LkjMJ3mIamLsUhNe7eU72Er+Y9P368O1VCYYq4SHrsbEgZ70+RkD2VdIaa\naD59NFQGJ0XoiL45fQZo3ywSrp8v4eTkGI4h0dnePkx/EKrkuHtP9RK+eskFZZZEF0mPnQ0p4/36\n52OPJqaWzhBL2iV8pofb9DJ82sVIMeQI5/TI6eTkIsHekl9bu9uKp1ByLu8o6T3VS/hPPNGN2MZG\nZ+FTSsp83p2XHjKfYmYDZbx/rG4plVNJ2PrIlTl8cvz09yUwSEZDbQ8w0GFSyrxXRwwNJ4WSG3tH\nyTGol/D7EdvZob1gM4dF26875DSfUiknbvAhcpd2bW8f9SIZJGejtAe4IJeYcbFfUoyBiyNaL+Gn\nGrGUEpE7eEcViuDIEDb4MIdLu554gh3hU6N0xCqnDRSjWHL0U2gZrtRSL+GnMgVSmhqplIlJSkqu\nDJUGlYLq/f+e7E+enFw/lhaTEuXHLBulrGdM9NCVWuolfN+bgl2Q2tSgCMC6EjsX37VmqH24tjZJ\n6760mJQu3xU56hkTPXSllnoJf6jVoQd5cm0TCL0SyIfYU3oqY0q1dIyACrY+nEr7ZPklmtLluyJH\nPWOXi1yoZVqEH6OGuZsaIcSeamVoSDGWjhFQQ+/DqbVPll+icdnVwkG/pu6nPqRj2jJKhfoJX5WG\n2BUZzqZGbmK3YUwxclecsZh6+5ihVIy/lIJJPZXrJnyTNMT0WGlTZwwu9UstrWOKkbvijEWm9tmG\nkYu1mwu5rzQcUjBT6Pu6Cb9ZW0fBZR2Cu+KMReL22YaRWzQptVLqQxyx+W18MBY55dL3oaib8LlZ\nk6VNgKYAj6L0eATCNowxeeD0TBO+yWBN/0+tlFw2SVEPsY1SpjK16iZ8KflYkxxMAG4K0AepZi4D\nk8y3aUObg3yHV+8Gl4zULl1HrZR8+sGnnqHl2lIv1Ti1VNRP+FwwJum5rE0uCtAHKWYuFfNEjttY\n04bCIrZUQT7Dq3fDpUs0yWApldIQhtqb2+qucWrpaITv4rv6XLRh20XDxNpkiRQzl4J5CMZNv1jj\n0qW8MXm9G3oLPyYZbD8lbOcTcxEjF6u7psjhahO+i/nlMyNtkj6VAGAqpJq5sczjk4jNMuNVEdKT\nuubMEKkfJVCTqaq7mtWfY5KR+tYv5nB86TMEqffOU2K1Cd93T/nOTlxmo40NKU+dkvLxx/1vrpg6\nSs9cE1wUkcqAs5n86ye25Ze2F3c5eKZQSmkLVav6nTwuQ3WhVlK1O78Mk6kOYrUJ33VVaGPDfUYM\nlfXkk1Lee2/XhcePN9KvAWOKSGHAHwLyb7Eun8ND8h2zu0nfFvMupef0cFN/29P6up24XJWUq9Ve\nu/PbCL8mwpfycMbZ7sXt/z92AsRFwi9ePCodly/TtaMm1BT0HEPPgGtr8ocH4/o9bMh3re0aRYST\nE6PbMydPHormUHhirB0+VntpLycWfUiHyf04o2BJ+AAeAfB1AN8A8BHD/w9bQEEeLhJKsSg7n3eW\n/Spb+DX48L4ytVhIubMj90/N5PexYbTwQ16dQy+q5E1xp6uU/lZ7qCLM1T8uZxY4KfIhsCN8AMcA\nfAvAAwA2ADwH4K3aM13tqcjDZ3EudlF2Pu8s+5JkTzVTQt7D3YePkanFQv71zq58esdO9r6Wb+4c\nMj7Wtm34c1jtOfqnBtvEFxwJ/wyAzym/P6pb+eSpFWIltCa/lEqKQ98z5ikRm2zer0yokHxeXUov\nulqrY8Of2urN0T8cbBPqKcGR8H8GwCXl958H8AntmcPeoCLaWAll4teNCgiVFMe8x9RXi4Xcnz0k\nb6+vy/0ZjdIM0kkJlbfPq1nZEAahKk2GOb0IDruoqMoPIXzRfS4NhBA/A+D9Usp/ffD7zwN4WEr5\nb5Vn5IULF7pfvv99nH/zm3H+F34B2Nz0Kmu5BPb2gNnM+6MssVwCZ88CN24Ap08D164Z2tU/dPMm\ncOqU5SGPwmLfc4D/t7OLe957DhvYx99iAz/YuYpXv+dM8PsAYHcXOHcO2N8HNjaAq1eBMy6vXC4P\nO5FYMHxenbAa7rAIFfHwB1ctdf+QlmEinAESCpZfBVeuXMGVK1fu/P7YY49BSim8XuKrIXy+0IV0\nPq/8bg/pRGCK8Tlnq4vKGyH0ar60vZDP4SH5vYNFzz4OHuPOlrbQ7qpMjTuTBo4KJ3Fqc/dTrvJM\nhKP+bTbrVs2VeqSQXzAM6azhcNH2HnSLtie1Z6IbHuKSctthYSqTDcF5YrGQ8h2zhXzX2q58x2zh\nlOTL9b3FI20crItQgVTrrh8VTlHHnP2UszwT4ah/61OCavWgll92hN/VCY8AuAXgmwAeNfw/uuEh\nOxJc99uWnN8sCC4Qat05xIjJFDbVie1QxArkYuGWdS0WuQc9Z3kmwlHHpSf9xPVgSfijFSA6eOVD\njj4n6kqTVc3oidYlyVfKOpAqbHWyx57YDgGFQNa0SuqqrXO7xJbNCnJnp5OJDPVYacL3gQ/h1xxa\nMaFUmNOWjTE1kijsfrLnvrOvL5uKSFMPSEgZqoD6auuYNlEsMo1lqyNGI/wRqBanT1a8mkMrKkqH\nOSe3FlLKGihJbFQw1UMX0FwKlSJMViDuu/KEPyTLXCzOkigZ5qRauA2tS7KxDn15Se1XejubrR6m\n9ZEQherbt7ETo1Dcd6UJf0yWWyy+bJgzVf9zMVi9UIp4uUyCsdvF9cVQH4U6n0u5teXXt7ETo5Cn\nt9KE73Ir4ZRi8aEoFZ5K0f9cDFZvlMy1wGESDNUjNly1tSWDdsnETowCEyuE8JOetHWBEEJS1MHl\ntOByCTzzTCcNb3/7NE7kAvWcMqY+TUlxetEbFJ1d8mgriyO/ieqxu9v16+3b3e9bW8Czz/KeFBEQ\nQkB6nrSdDOED4zLklK6gAqicA0yjTSFIzps6uVMKEBfinRLU8XnLW4AvfhF4wxtK1yoZVp7wx1DE\nIiSGzjm//dvABz6Qvk1cvYhkvGki9729+gXIFVwHfAwrpEhDCP9YqspwwHLZkfxy2f0+m3VysLHR\nWYSnT/u/ozT29jp53t/vLFsh/Nvki577zp3rvnPpC6Cb02fOHJ3bJGOmd/SNG2ECVCM4D/gYTALB\nAVyIxDfoT/2FRPvwbQt6PmsrORcFYw4Upl4v4rK5wwVkY2ZbWJzKoYwh1DTgUubZqhVTRiIiwSrv\n0tExJLOuY5dL7lMdKKSaB1w2d7gg5Eo+0/mf69elXMxXgNxNcB1wDntic1hlIRNU7ZdERNIIX8GQ\ngeY6drmILvb+kaGr6dSDZv0p49BEizVwn8+YmWSB1EOgJsOcBDs24Fz2xOY4jesyQfuxMZ0wTEQk\njfA1mGSW+lJmijkYKg9Dc05vZ38W5fhxY+bWScFVOY1luT0iHz4DHUqGQ2VwIdgeHMI+i0WXG6Xf\nd++SJyW0nKEJqo7N1pb59vgEFlMjfAVjli9F7qntbbpkiSHyMBa26tupymCmzK1VwLYeclfqbF+y\nDb2ggctGl2A0AAAZSklEQVRR8THlpgv/1lZn2eaG2idra8NZEGMxNEFN1lXLlklD+C6G1tjciVW2\n6vtNBJrL83YxPHZ3j6YnTn3vRW3QZWGxkPLkSSmPHeu+LxYyzC30tSq4HBV3mTz9/0+dkvLEiXIC\nxWVxSa9HpkRdkyd8V0MrtTE0dLlNbyG6ZuKMhavyUsm/hlh8KRhTZ4cQi69V4VJGjoWUMatZvybR\nFL7ICS6LSwXqMXnCdyXy1Ipfff9s1s2JvgyfXPu5kHutr/TGjRhYxy/HhOZAXmNxcV34M1320XA3\nJk/4PkTuMnf6UKR237BzXUzv50b4Odf6uK0rhsAYw+cKX+3q+vzYzhdV+G0ToXbNXwEmT/hS0hlB\nIQv8rusHPvflpp4TOdb6+naUuAAqBTgY2qPw1a4usXn9pqnYdMEuZfm8symQI1gJwqfC9etHd66s\nrw8TlM/8cvUucljDucJbvYJrHn4m+Gpyly1d+t7xUK3nW9YYpuA6JkAjfANshoFu4Z86NSxH1JZy\n7l12qSxW0yVF7K1jB7A3KG2afEjgbZrf52CRD0GHlqWDw55/hmiEr2HMMJjPu11lx465hV8oLeWq\nYsUDCN3Awtmjr8ag1DW5SyjFFm93PVgUu0spVGA4bL9khkb4GsYMA99zG5SWck/4ubZvpoRPv4Rw\nR8itdTGo1qD02cZmSiDkerAotkNCJlIVCyt50Qhfg4vhkuNktgnVkkokQs4wqbfWpT5Q2ZdZpUHp\nUvGYGDrnDmEfg6NHI3wDxgyDVDtLxuSvhjmUAr7t1hfXXRQzxdyv1qAcq7iucS9dCgvRcCHYxYI2\nx0lFmDThp5IvFy8gJN7ssittPu+s1ZBzANygtsll66pvCEglfZdDd8XmPhcitEEV+NBMesU7WauH\nq3BMDJMl/NQxXBMBxRgOrrvS+m2Mvmm2Q1Mcp4I671Lk6lksOsXosuWzaKiMCxGOYbHoLPvQtAgp\nOjlEUeo5Trj3ewwM/TNJwtdjuDkmcazhMOQ1hKYiUbmEOsVxrFGqz7tU4+TiGSQLlbl0ko0IOVr9\nrvH+VCln1XeHKkq1HnqOkynB0j+TJHw9hvvGN6bPxEphONjISZdR14NKqUg11ijVPaHjx915IGWY\njjT+7tpJJiLkbPUPdZRLXDK0k/V3xyykVbvYMgB9YlgMiUkSvmpt33NPnnmjhpBSGA6qjLrKqx56\npeoH3eOwreGZyFkPT+3suGfjpOTBXukkWwvxCWHoA1oqxkTptlHX23RaL9Rj4Og9uUKvuy2ObPGo\nJkn4fT9cupRn3qhEVOpeBxt6LqFMcayHikwEbCPnmPsnqPhksUiztfbIXIwJYSSLMdkqK+0D5vvO\nkHq7ELDNE/JNwsbZexqDXnf1akSTC2/on8kSvto/nO+XrRVjCnUoND1GtinDwH3dfHIiucDII7Eh\njFRhB1NlKbVpSE7/XAmnap6spu2xtks2LJgs4fcyfOtW1y8pre5cioUbhto99L+h8GtoGNjHS09h\n4VfFI6bKlhLiXAmnegFRr3KrbbLqY6S2xTGOXC3huxxQctmdQhXOm+I6kAv0tQU9SuCbGiXF1a62\nz+zs0K21VKX0bZVNIcRjE4y642yhHz0UEtLOErH/sUnl6vUcvKNawh+a3K67U3yJIna8XWS/5rUk\n376kypPFxbp25ksOA53DQnEVCuq66O+jEJDeLcyZudA33DUUBz14R7WEPzR2KmkM7U7x3UjR7/wJ\nWZh1CVWEWKmleaPH9jbdVaWhYeCqrOsaFw3HoAukvp3r8cfLtJdCQHyupaOamD7b4WwypZFctYQ/\nNnY9aQx5bz5yoC/0bW35jadPFk7XBGFceCNFTDykDlWE1Li4IzaEkpVJIHXBKCUcff1iBMSV8Ckn\nph6bDrFcNZKrlvB98qqMhVFc94DHnN4dUy6+Rggn3tANkdJ38rIGZ3ckhqxsAqm6fv1OEm5KzgW9\n8hrLTR5iuY0R1Nj+chdyOSC5agnfBdRWcH+4KubaziHl4mOEcOINTnXxQbGQGFd3JMaKUIVgNjs8\n0cbB/QuFLiAu4+YzGXzWOIa2w/W7jxxkatKEnypfE5e5SlUXCuLj1C8u4BQSY4NYzb1YHGas00M7\nlFuiciBGQPr2jh3j9rl8Znv77v4LqOOkCb9WyzMlTEZLLzO9YZYisyanBWYpeYXEWCFWc0+lYym8\nnRjL3eVdAXWcNOH3/VWT5ZkSJtnRt7AeO0afrpijNV3MGOCm+UJha8dUrKyYdoTmUTL1qceC7Mpb\n+EOYyrzzwdABS/3cAqWBxtXoy24McNR8PYYmxJBb6HMcOld9KcsIPaDlQ8R9yMZ00YXHgqwLVpLw\nOc87Kpjmg0129NCrT7pi17pMweiLBmfNZ5sQY25hiXZQT2Af5eH6rCsR920Zukhj7F0e9V9JwrdZ\nulOx+Mfm79C5BOrMmi7lrgy4ar4hAueUd8elvr7wUR4pLEVTJj+fd3vWfyUJX5dXNcsop3noClVZ\nLRZSXrzoF0Kc+gItK3DSfP1ADSUUG3ILS7WDUuH4KI8Uns183rnUgJT33ivlk0+mWS846LOVJPy+\n/aplz9HTdoGq4PvbsPp7Yl1OI1MrulUIl1WFscXVfqCG0sqmJPdQ64CqTj7KI4VnE0s+rnU6KIcV\n4QO4AODbAJ49+HrE8pxfp4ygtIcaA/1CEfXE6+XLw21Joehid7Nx8Ay41CMaQ9pXPx7dX9c2ZiFQ\ndgwX68BHeVArP1O4wbePXerE0cI/IPxfc3jOvTMcUdrTjjF01AOOrvfd6p8tvUDLae5zqAcJXLf0\n9WTvkpGQOn5dq2tNCXXxLKaPx0iEWwz/gPB/3eE5v47IhFjSjhln3/tuTZ+lQsg7ucx9knpwcRHG\ntK9ONKY4fd+OVMfWQy0OLn1MCXUBd309/LCXmtpCA0fC/3MAzwH4PQCvsTzn3hGZEEPaXMiuJLiE\n1aLrwc1FcNW++nN6O1LdFBViHXDrYyqoC7jHj/vlYNdPUFr6JoTwRfe5MAghtgHcp/4JgATwGwC+\nBOD/SCmlEOI3Afy4lPJXDO+QFy5cAAB8//vAm950Hr/4i+exuRlcrWjs7gLnzgH7+8DGBnD1KnDm\njNtnl0vg7Fng5k3g1Cng2jUUbUspLJfAjRvA6dNl2x9VjxhBMFVkbw+YzfJ3iKkdp0/zGCDKPu5R\nsq97UJDI3h5w+3b3t40NXPn4x3Hlu9+989hjjz0GKaXwqpevhgj5AvAAgK9a/iel5KXoYy3DUmsI\nU/SMKRDcL76C4LqLpoRgxC4m5qob1U6d0kRCQSL9CUrLO8AspPN65ed/B+DTlueklPxCIaUXfn3B\nRc65IbpffMIoLrtobDnQc6UW4HhQhXKypSKSkDGiaNfAO7gR/n8B8FV0Mfz/DuA+y3N32tXLYci1\ng1ODr3xxU5gcsFh0B9eormscRGhirNyaunZBcdi9kmy7mp4murCXxIrwnSugLNr2l5JwMj5KIIQD\nuCyUcoHah9QZQwcL9E2MlZuAaxYU14mhejMxpNyT+vb20THa2WHhJVVP+FM46EMBl34wtbe2MFRK\n6H04dnCNBDG7VHIScK2C4puq2EbKLmShb41UY+m6AijkJVVP+KGyzzl+HRr6GzMWubaXC6oyZFMR\n8GLRkdPYbU21wGdQbcrBdfLon9/ZOXpAhoFwVU/4UobJPtewZAwxD/UD1/ZKycvTGpMlTnUlx2JR\n7/2zQ/BdRNdJ2TNBGVXu+hSYBOGHgInCvQspNwxwbG9NnkdNdQ2CfnhnbY2PZeCy8EqhiU2k7DN5\nGJD6EFaW8KXkOTYpiZljezntiBsDZy/pCFwbrz/H1cIf07SUmtjWdxwnTwBWmvC5IoVs2RZsqS/6\n8X1PCgWXyhLn6iUdgc+uFNNzi0UXe97Z4dPAMU1LpYlLuHCZY4SN8FcAJjn2kW2qeTDEMZQKLqUl\nzt7Qc218Ne6KdN+REKuJKRWHq4eVWcFMivBDd7eEKthaFvBMclziop9cHJPbEmclB66Nr8JdUdBr\nWts+eQpN7KJYfLZmhu4KSoiqCV/t/xBlGaNga1rAM8mx7zoUBTfk5JhclnhyOQi1YkIyZHJHjkln\n6xPXsnUSv3SJldKtlvD1/vc519DPoZizEDV4xLpCNG0+yH3RT20cM4akclCTVREDV6VWctKFbM08\nfrzb6eRywrfF8IcJ33TGwceTNR2GC7HwY7y/lKidK0r3nyuSGmn6NYRD1qILOHZqyGJSbCbSmHq6\nusSXLmVKyOSHagnfFqYYU5ZDh+F8Eev9UUOV7xo8EBtqU1bJjDS1I2KT+3DtVF9Btbmq+tav7e3O\nmqNsr69LzHCNpFrClzJsouUYh5yLk7Y1jFQXFLnWJwbclFVRw7i3FvUO8a1UCaF0fT5GUG2C31vX\nORPMVZCoqmrCD0XqcbDJcAoPc2gNI6e82QzI0DVHLsYRC8NY7xCf/PT9AOSwAEI7K0ZQTYuk+lV/\nqXOnsxASN6wM4ee20nQZppaJ0DWMVDAZkDFt5mIcsfE21A5RL7teW+sG3/YZ3fpN2aklOsumDDc2\npDx5UsoTJ8YXTmPBRkjGsRKEz0EBh8qETVGFrmGkUnym+vhsbOC2ltiDk7dxB/O5lPfee2jFnjxp\nrtiq5M03WVe7u/lSErMUEjNWgvA5KOAQmRhTVL5WcCrFp0YNTF7NUJs5KOMxcPE27kC18IeSnJUg\nIk6dlbP9nNo9gJUgfC4K2FcmqBVVCsUXq5Q4KOPqsFi4JznLQUTcXbQKiDgXQghfdJ8rByGE9K3D\ncgncuAGcPg1sbiaqGDGWS+DsWeDmTeDUKeDatbi6U78PAHZ2gEceAW7fBjY2gKtXgTNnytZpJbBc\nAs880/388MPlOq0fwH5ytQFkDSEEpJTC6zM1En6toFZUse9bLoG9PWA2635/5zu734Hub9ev29+r\nflZ9hqqNtvc3JMTuLnDuHLC/H6bxG7KiEX6DM5bLjuCffx44eRL42MeAD3ygm+vr68DnPw+85z32\nz6Y0BJuhWQhjLlrTwqwQQvjHUlWmgTeefrqbu7dvd9//5m86ct3Y6L4//LD9s3t7HRnv73fccOMG\nbd1M718uOwN0uaQtq0HB5mZH8levmsn+7NnOAzh7Fnj55TYgvggRYmLBnzThT50kKNv3qlfZ57qO\n2exQOZw61f1MCf39999/lGumOp4sBHZzswvj6AKga+F3v/vuAeFQf67QFaZLH4V8Zgy+q7zUXzhI\nnpZqL3nIQcGY26RyIXYLZL85pE88F/L51Cec1bNJ7Hf/xApJ6IDkEk51e9zW1t3JxEruyc05QUPL\nChFi9TOGQ3modVvmkIzk7F+TzLrKcW6lQEGCtexy47IV1woKstvePtyaCdhP3FKX64NeYEypHUpp\n5Zx9EFKWerAl5PDOwJbdagnfJiMxY+lDEv2YqIf5eoXqIscljJtS53BKeTpDyqm4B0ZBdiGEX9L1\nMZ2ILaGVc/ZBSDbQ2HQYAyeMqyV8m4zEjqWLBauOyWzWnWxXFaqLYi5p3JROqFYaPvUiVwwx1pvp\nXbNZZ2moltxQpbm5PiVcxtwncH3KonLD9fxCB/JQLeEPWW6px1Ifk098wj9TZS6ZK2nJjsluqbr5\nXl5EprAorDfTO32z9HGPy+UQjNyWT2wefd8+0UNpB/JQLeG7tDUlidoS9OXKCutTz1IW9pBSK71e\n5zJe5F5YDreuitXqAZQWWg4IUeI2aPIwScLPAdOY9AqV2jgJNXhyzn1bHW1KrTQv+YTuyLywPgQT\nus3JtQxOIRtfcBBabojpE00eGuETIoVxkmsROncdqeqWes6SemG2mDs1cocrKAeAs9CWQmyfKPLQ\nCJ8QMTFr2/9yLELHIibXf0zdapqzUkrzrTU1HOCwYWgA1HaExp85Cm0pEPVJI3xChMSsF4vh+5Zr\n8NDH6ugz332erW3OHumo2ezuQa9Ng9kGQG2HqZ2uSKn8aphYCdAInxg+Mete5sbuW87poYfCVMcx\nZWZ6hw/fVTln+44y7ZWuTYPZBkA/7Rnq/qVWfhQuZi3e2AFWlvBdx4pqTE1zQ50X/X3L1RDXCFyV\nmYrQk87cleER9AOvbu3qL9muUYPZNL3uyfi2ibvy4+yNDZDWShK+7nFub9vj6pRjqs8NfV7s7Bwt\nYz6X8uLF7nttCFFmNfKdF3SBunXrMMeMGtapSoNZoLYjpE3chYGrQhohrZUkfFcySjWm+nqWaS7M\n51IeP96Vffx4faQ/psyGPjcFvjNCF6hLl3iSRk6M7WTwvbQ5Z0I0DgpJb7N63/H6+l0ytZKE7xM7\npx7T+bwz6sYs3osXD+sGSHn5cnzZFPBdgJ0CeZPxiC5QVOkVKosj3wGlC10ixFJawE1tHrEUJ0P4\nvnK/WHRW51h4kXJMF4uO7F1i2hwtfM5hyx7U/EfeZlNcL1TAahiQIVC60FxDLCqohdPU5pF+mATh\nx8h9T/y2OD4lVG8L6Mh/qMz5vLPsOZC9lPznVAr+Y91m1pVzAKULzSXEYkMK4TSFC4b6YTGRg1cU\nJ49zp8buN2bUBO5zKgX/mdpcPIpi2unDcUBcQGlxlQ6xDIFaOIfIZGDn1CQIP4aIchhJLou0OUBB\nVJznVCqFpG84KRpF0StAkXGzJFJ3aHHtrNSDUjh9ievg+UkQvpThRJTaai1OEAT14DJnXJBaIRWP\nohSvADFStofD5Etl7fkS15QsfJ8+MqUu2d523zboCy7zMybfTek5wwnFw1rFK0CMlO0pPflyeC9j\nl26ohDeVGP5QG9W/632fg8xi5ZnKug6tR+k5wxHFw1qpKlDKlXNtj2/9SivHocmTuq8t5DYpwh8i\n8IAdTGSIDTelOunrU4epGJQNFrgIW8nY3lj9hiy9kotmpsmTw9K0kFt2wgfwswD2ANwG8Dbtfx8F\n8E0AzwN438A7fNoopTT3PXcyo7recsoLtSpyGE21rGV4wyW3d8nYnsvk5hh3NE2eXDtFDORWgvAf\nBPB3AfyJSvgATgL4CoB1AG8B8C0AwvIOnzYe+b9pt5KJzDhMbopwkMs8eOqpp6LrSomQvs8RLnV9\nf/b+dOmwsWfGhK1QbO9OXw7VT63b+nq3IMcZuSxNA7kVC+kAeEoj/EcBfET5/XMA3m75rE8bg/qJ\ni8EQ0x7XOXrhwoWoOlIitO9TbHPWU5S4vj9rf7qGYlw6dUjYhkgqoXV0pC+HrLPZTN450ZjyNjEq\nFHKbQwj/GNLgjQD+Qvl9fvA3L2xuAmfOdN9DsbcH3LgB7O8DN292P5dCTHtmM+D0aWBjAzh1qvuZ\nO0L7nrKtyyVw9ixw7lz3fblk3JcuHebaqUPCtrkJXLsGXL3afe+fMXVWKtjqt7kJfOxjwNpa9/ut\nW2UnrQnLJbC7e9g/FESVCaOEL4TYFkJ8Vfn62sH3f5yjgrFgO7k9YZujnBHa95RtNfEj27506TAq\ngTaRFBfr6O1v79rJcdLmVIoJIDrPIPIlQjwF4NellM8e/P4oOnfjPxz8/nkAF6SUTxs+G1+BhoaG\nhhWElFL4PL9OWLZa8JMA/kAI8TF0oZwtAM+YPuRb4YaGhoaGMETF8IUQHxRC/AWAMwD+hxDicwAg\npbwJ4L8CuAngswA+JClciYaGhoaGYJCEdBoaGhoa+CPVLp1RCCF+VgixJ4S4LYR4m/a/jwohvimE\neF4I8b5SdawVQogLQohvCyGePfh6pHSdaoMQ4hEhxNeFEN8QQnykdH1qhxDiBSHEnwkhviKEMIZ3\nG+wQQnxSCPGKEOKryt9eK4T4ghDilhDij4UQrxl7TzHCB/A1AP8EwBfVPwohTgL4Z+gOb30AwO8K\nIVqc3x+/I6V828HX50tXpiYIIY4B+M8A3g/gNIB/LoR4a9laVY8fAjgvpfyHUsqHS1emQnwKnTyq\neBTAjpTyQXSHXz869pJihC+lvCWl/CaOLvYCwE8D+IyUcl9K+QK69AxNQPzRlGQ4HgbwTSnli1LK\nHwD4DDq5bAiHQFkDs2pIKf8UwF9pf/5pAL9/8PPvA/jg2Hs4DgDJoa0GfFgI8ZwQ4vdcXL2GI9Bl\n8NtoMhgLCWBbCPFlIcSvlq7MRPA6KeUrACCl/EsArxv7AOW2zLsghNgGcJ/6J3QD/xtSyj9KWfbU\nMdS3AH4XwL+XUkohxG8C+B0Av5K/lg0Nd/AuKeV3hBA/ho74nz+wWhvoMLoDJynhSynfG/CxOYA3\nK7+/6eBvDQo8+vYygKZc/TAHcL/ye5PBSEgpv3Pw/X8LIf4QXdisEX4cXhFC3CelfEUI8XoA3x37\nAJeQjn5o6+eEEPcIIX4CA4e2Gsw4GPwe/xRdCusGd3wZwJYQ4gEhxD0Afg6dXDYEQAjxKiHEjxz8\n/GoA70OTyRAI3M2Vv3zw8y8BeGLsBUkt/CEIIT4I4D8B+DvoDm09J6X8gJTyphCiP7T1A7RDWyH4\nj0KIf4BuZ8QLAP5N2erUBSnlbSHEhwF8AZ1R9Ekp5fOFq1Uz7gPwhwdpVNYB/IGU8guF61QVhBCf\nBnAewI8KIV4CcAHAbwH4b0KIfwXgRXS7G4ff07i0oaGhYTXAJaTT0NDQ0JAYjfAbGhoaVgSN8Bsa\nGhpWBI3wGxoaGlYEjfAbGhoaVgSN8BsaGhpWBI3wGxoaGlYEjfAbGhoaVgT/H78HTBbmqjq7AAAA\nAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7ff51bc08fd0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "%matplotlib inline\n",
    "\n",
    "#declare an interactive session \n",
    "#and convert tensor to numpy arrays\n",
    "sess = tf.InteractiveSession()\n",
    "xor_features_np = xor_features.eval()\n",
    "xor_labels_np = xor_labels.eval()\n",
    "\n",
    "#We show only a part of the dataset\n",
    "portion = 200\n",
    "chunk = dataset_size\n",
    "interval = chunk*0\n",
    "plt.plot(xor_features_np[interval:interval+portion,0], xor_features_np[interval:interval+portion,1], 'b.')\n",
    "interval = chunk*1\n",
    "plt.plot(xor_features_np[interval:interval+portion,0], xor_features_np[interval:interval+portion,1], 'b.')\n",
    "interval = chunk*2\n",
    "plt.plot(xor_features_np[interval:interval+portion,0], xor_features_np[interval:interval+portion,1], 'r.')\n",
    "interval = chunk*3\n",
    "plt.plot(xor_features_np[interval:interval+portion,0], xor_features_np[interval:interval+portion,1], 'r.')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Save in TFRecords format\n",
    "-----------------------------\n",
    "\n",
    "We can use the numpy arrays in order to store the dataset in a TFRecord file. First of all we have to create a function that is able to convert the array in a serial file. Then we can use the function on our dataset and store the files on disk. I will split the dataset in test and training set. The test set is 1/5 of the total dataset. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def numpy_to_tfrecord(features_array, labels_array, output_file):\n",
    "    with tf.python_io.TFRecordWriter(output_file) as record_writer:\n",
    "        for i in range(labels_array.shape[0]):\n",
    "            #Getting the data as train feature \n",
    "            float_feature = tf.train.Feature(float_list=tf.train.FloatList(value=features_array[i].tolist()))\n",
    "            int64_feature = tf.train.Feature(int64_list=tf.train.Int64List(value=[labels_array[i]]))\n",
    "            #Stuff the data in an Example buffer\n",
    "            example = tf.train.Example(features=tf.train.Features(feature={'feature': float_feature,\n",
    "                                                                           'label': int64_feature}))\n",
    "            #Serialize example to string and write in tfrecords\n",
    "            record_writer.write(example.SerializeToString())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "\n",
    "#merging for shuffling\n",
    "xor_merged = np.concatenate([xor_features_np, xor_labels_np], axis=1)\n",
    "print(\"Checking shuffle consistency...\")\n",
    "print(xor_merged[10]) #printing a random value before shuffle\n",
    "np.random.shuffle(xor_merged)\n",
    "print(xor_merged[10]) #printing the same value after shuffle\n",
    "print(\"\")\n",
    "#unmerging after shuffling\n",
    "xor_features_np = xor_merged[:,0:2]\n",
    "xor_labels_np = xor_merged[:,2:]\n",
    "\n",
    "#splitting in training and test set\n",
    "to_take =  xor_features_np.shape[0] - (xor_features_np.shape[0] / 5)\n",
    "xor_features_train = xor_features_np[0:to_take,:]\n",
    "xor_labels_train = xor_labels_np[0:to_take,:]\n",
    "xor_features_test = xor_features_np[to_take:,:]\n",
    "xor_labels_test = xor_labels_np[to_take:,:]\n",
    "\n",
    "print(\"Train features: \" + str(xor_features_train.shape))\n",
    "print(\"Train labels: \" + str(xor_labels_train.shape))\n",
    "#random_indeces = [200, 2845, 5009, 7985]\n",
    "#print(xor_features_train[random_indeces])\n",
    "#print(xor_labels_train[random_indeces])\n",
    "print(\"\")\n",
    "print(\"Test features: \" + str(xor_features_test.shape))\n",
    "print(\"Test labels: \" + str(xor_labels_test.shape))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "numpy_to_tfrecord(xor_features_train, xor_labels_train, \"./xor_train.tfrecord\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "numpy_to_tfrecord(xor_features_test, xor_labels_test, \"./xor_test.tfrecord\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Conclusions\n",
    "-------------\n",
    "\n",
    "The TFRecord files of the XOR dataset can be easily used in a Tensorflow model. The features are represented as a list of floats whereas the label is a value that can be zero or one.\n",
    "\n",
    "**Copyright (c)** 2018 Massimiliano Patacchiola, MIT License"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 2",
   "language": "python",
   "name": "python2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}
